- 
                Notifications
    You must be signed in to change notification settings 
- Fork 25.6k
Create SPI to allow prohibiting certain top-level mappings #132360
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Merged
      
      
            quux00
  merged 20 commits into
  elastic:main
from
quux00:cross-project/prohibit-_project-mappings
  
      
      
   
  Sep 19, 2025 
      
    
                
     Merged
            
            Create SPI to allow prohibiting certain top-level mappings #132360
                    quux00
  merged 20 commits into
  elastic:main
from
quux00:cross-project/prohibit-_project-mappings
  
      
      
   
  Sep 19, 2025 
              
            Conversation
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
    16994be    to
    1ddcf5a      
    Compare
  
    
              
                    javanna
  
              
              reviewed
              
                  
                    Aug 21, 2025 
                  
              
              
            
            
              
                    javanna
  
              
              reviewed
              
                  
                    Aug 21, 2025 
                  
              
              
            
            
        
          
                server/src/main/java/org/elasticsearch/index/mapper/RootObjectMapper.java
          
            Show resolved
            Hide resolved
        
      
              
                    javanna
  
              
              reviewed
              
                  
                    Aug 21, 2025 
                  
              
              
            
            
        
          
                server/src/main/java/org/elasticsearch/index/mapper/RootObjectMapper.java
              
                Outdated
          
            Show resolved
            Hide resolved
        
      1ddcf5a    to
    64eddb6      
    Compare
  
    1. One case with subobjects:false doesn't work as hoped:
// this does NOT return an error - it just silently "fails" to create the _project entry
PUT test333?error_trace=true
{
   "mappings": {
     "subobjects": false,
     "properties" : {
      "_project" : {
        "type" : "object"
      }
    }
  }
}
// YAY - this one fails with correct error message
PUT test333?error_trace=true
{
  "mappings": {
    "subobjects": false,
    "properties": {
      "_project": {
        "type": "object",
        "properties": {
          "myfield": {
            "type": "keyword"
          }
        }
      }
    }
  }
}
2. Creating a runtime mapping is not detected and prevented:
PUT /rt-index
{
  "mappings": {
    "runtime": {
      "_project": {
        "type": "keyword",
        "script": {
          "source": "emit(doc['existing_field'].value + ' some additional text')"
        }
      }
    }
  }
}
    …ected - basic manual tests now pass
…ssField can call it without a Mapper reference
This prevents _project or _project.foo mappings in query-time runtime mappings.
For example, these now fail with "Mapping rejected: [_project.foo]. No mappings of [_project] are allowed in order to avoid conflicts with project metadata tags in serverless"
GET /blogs/_search
{
  "runtime_mappings": {
    "_project": {
      "type": "keyword",
      "script": {
        "source": "emit('somevalue')"
      }
    }
  },
  "query": {
    "match_all": {}
  },
  "fields": ["_project"]
}
GET /blogs/_search
{
  "runtime_mappings": {
    "_project.foo": {
      "type": "keyword",
      "script": {
        "source": "emit('somevalue')"
      }
    }
  },
  "query": {
    "match_all": {}
  },
  "fields": ["_project.foo"]
}
    …d.parseRuntimeFields. Removed checks from RootObjectMapper and IndexService. This checks both query-time and index-time runtime field mappings.
…ime; no query time runtime mapping tests
64eddb6    to
    af4964f      
    Compare
  
    
              
                    javanna
  
              
              reviewed
              
                  
                    Aug 27, 2025 
                  
              
              
            
            
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
solid work, I left some minor comments.
        
          
                server/src/main/java/org/elasticsearch/index/mapper/RootObjectMapper.java
              
                Outdated
          
            Show resolved
            Hide resolved
        
              
          
                ...c/main/java/org/elasticsearch/index/mapper/ServerlessRootObjectMapperNamespaceValidator.java
              
                Outdated
          
            Show resolved
            Hide resolved
        
              
          
                server/src/test/java/org/elasticsearch/index/mapper/RootObjectMapperTests.java
              
                Outdated
          
            Show resolved
            Hide resolved
        
              
          
                server/src/test/java/org/elasticsearch/index/mapper/RootObjectMapperTests.java
          
            Show resolved
            Hide resolved
        
      | Pinging @elastic/es-search-foundations (Team:Search Foundations) | 
              
                    javanna
  
              
              approved these changes
              
                  
                    Aug 29, 2025 
                  
              
              
            
            
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
              
                    ldematte
  
              
              reviewed
              
                  
                    Sep 1, 2025 
                  
              
              
            
            
        
          
                server/src/main/java/org/elasticsearch/node/NodeConstruction.java
              
                Outdated
          
            Show resolved
            Hide resolved
        
      …it-_project-mappings
…it-_project-mappings
…it-_project-mappings
    
  szybia 
      added a commit
        to szybia/elasticsearch
      that referenced
      this pull request
    
      Sep 19, 2025 
    
    
      
  
    
      
    
  
* upstream/main: Turn NumericValues into functional interface (elastic#135068) Improve block loader for source only runtime fields of type keyword (elastic#135026) Mute org.elasticsearch.xpack.esql.qa.single_node.EsqlSpecIT test {csv-spec:stats.StdDeviationGroupedAllTypes} elastic#135103 Mute org.elasticsearch.xpack.esql.qa.single_node.EsqlSpecIT test {csv-spec:stats.StdDeviationWithLongs} elastic#135102 Mute org.elasticsearch.xpack.esql.qa.single_node.EsqlSpecIT test {csv-spec:inlinestats.StdDevFilter} elastic#135101 Mute org.elasticsearch.xpack.esql.qa.single_node.EsqlSpecIT test {csv-spec:stats.StdDevFilter} elastic#135100 Remove track_live_docs_in_memory_bytes feature flag (elastic#134900) Create SPI to allow prohibiting certain top-level mappings (elastic#132360) Only validate primary ids on release branches (elastic#135044) Added no-op support for project_routing query param to REST endpoints that will support cross-project search (elastic#134741) Fix race in FileSettingsServiceIT.testSettingsAppliedOnStart (elastic#134368)
    
  elasticsearchmachine 
      pushed a commit
      that referenced
      this pull request
    
      Sep 22, 2025 
    
    
  
    
  gmjehovich 
      pushed a commit
        to gmjehovich/elasticsearch
      that referenced
      this pull request
    
      Sep 22, 2025 
    
    
      
  
    
      
    
  
`subobjects:auto` is guarded behind a feature flag, not enabled. elastic#132360 added tests without proper guarding. Fixes elastic#135134
    
  DonalEvans 
      pushed a commit
        to DonalEvans/elasticsearch
      that referenced
      this pull request
    
      Sep 22, 2025 
    
    
      
  
    
      
    
  
`subobjects:auto` is guarded behind a feature flag, not enabled. elastic#132360 added tests without proper guarding. Fixes elastic#135134
  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment
  
      Labels
      
    >non-issue
  
    :Search Foundations/Mapping
  Index mappings, including merging and defining field types 
  
    Team:Search Foundations
  Meta label for the Search Foundations team in Elasticsearch 
  
    v9.2.0
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
The RootObjectNamespaceValidator allows prohibiting "namespaces" or top level fields in
Elasticsearch mappings: both "normal" index mappings and runtime mappings.
The same validator is used to check both index-time runtime mappings and query-time index mappings.
The primary target use case for this SPI is a feature in serverless to prohibit mappings like
_projector_project.my_sub_field, but the mechanism provided in this PR allows anypossible namespace detection and prevention.